This invention relates to apparatus for reproducing multiplexed data from a record medium, such as time division multiplexed video and audio data recorded on an optical disk, and more particularly, to such apparatus which senses synchronization errors and controls video and audio decoding as a function of those errors.
It has been proposed to record digital video and audio data in a time division multiplexed format in successive tracks on a record medium, such as successive circular tracks on an optical disk. The multiplexed data is arranged in a pack, such as in accordance with a suggested standard (ISO1172) of the Motion Pictures Experts Group (MPEG), and as schematically illustrated in FIG. 7, one or more packs are recorded in a track. A pack is comprised of a pack header followed by a video packet and an audio packet. The pack header includes a PACK START CODE which, preferably, is in the form of a unique bit pattern which identifies the beginning of the pack header. A system clock reference SCR follows the PACK START CODE and represents a time code corresponding to the time at which the pack was recorded. The time code may represent real time (hours, minutes, seconds, etc.) or may be a multi-bit number derived from a system clock that was turned on when the information on the record medium was recorded. For example, the time at which the last byte in the preceding pack was recorded on the record medium may be represented by system clock reference SCR.
The pack header also includes MUX RATE data representing a transfer rate, that is, the rate at which video and audio data included in the video and audio packets, respectively, are time division multiplexed.
The video packet includes a packet header followed by encoded digital video data. The video packet header is comprised of a VIDEO PACKET START CODE whose function is similar to that of the aforementioned PACK START CODE, namely to identify the beginning of the video packet. It is appreciated that the VIDEO PACKET START CODE thus is formed of a unique bit pattern that may be easily recognized. Following the VIDEO PACKET START CODE is a video decoding time stamp (DTSV) which is a multi-bit number that identifies the time at which a video decoder initiates its operation to decode the video data. As will be described below, in a proposed reproducing device, the video and audio data are demultiplexed from the pack illustrated in FIG. 7, and supplied to respective video and audio decoders. It is important that each decoder be supplied with a START signal that coincides with the beginning of the video and audio data, respectively, thereby assuring that the video decoder properly decodes the video data and that the audio decoder properly decodes the audio data. The video decoding time stamp DTSV is referenced to the system clock reference SCR, the latter being used to preset a clock which, when incremented to a value equal to the video decoding time stamp DTSV, essentially turns on the video decoder.
The audio packet is similar to the video packet and includes encoded digital audio data which is preceded by a packet header. Like the video packet header, the audio packet header includes an AUDIO PACKET START CODE which, when detected, identifies the beginning of the audio packet, followed by an audio decoding time stamp DTSA which represents the time at which the audio decoder is to be turned on to decode the audio data. Like the video decoding time stamp DTSV, the audio decoding time stamp DTSA is referenced to the system clock reference SCR.
One example of apparatus which has been proposed to reproduce the multiplexed video and audio data exhibiting the format shown in FIG. 7 is illustrated in FIGS. 6A and 6B. Assuming that the multiplexed data is recorded on an optical disk, the reproducing apparatus is comprised of a disk drive 1, a demodulator 2, an error correcting code (ECC) circuit 3, a ring buffer 4, a demultiplexer 5 and, as shown in FIG. 6B, separate video and audio decoding channels, each being comprised of a buffer and a decoder. Disk drive 1 is conventional and includes a pickup head which reproduces the data packs having the format shown in FIG. 7 and supplies same to demodulator 2 which demodulates the multiplexed data therefrom. This demodulated digital data is supplied to ECC circuit 3 which, as is conventional, detects the presence of errors and, based upon known algorithms, corrects those errors provided, of course, that such errors are not so pervasive as to be uncorrectable. The error corrected digital data then is supplied to ring buffer 4 which stores such data until a predetermined amount is accumulated, whereupon the buffer supplies the stored data to demultiplexer 5. The ring buffer thus provides a buffering action to the operation of the ECC circuit, which may be variable, thereby supplying a substantially steady stream of data to demultiplexer 5.
The demultiplexer includes a data separator 21, a video decoding time stamp register 22, an audio decoding time stamp register 24, a clock register 26 and comparators 23 and 25. Data separator 21 acts to separate the encoded video data, the encoded audio data, the system clock reference SCR, the video decoding time stamp DTSV and the audio decoding time stamp DTSA from the multiplexed bit stream supplied thereto by ring buffer 4. The separated video data is supplied to and temporarily stored in video buffer 6 and, similarly, the separated audio data is supplied to and temporarily stored in audio buffer 8.
Clock register 26 is preset by the separated system clock reference SCR and is coupled to a clock generator 27 which, in accordance with the MPEG standard, generates a 90 KHz clock signal. Clock register 26 thus is incremented from its preset count by each clock signal generated by clock generator 27, thereby producing timing data referred to as system time clock values STC. The system time clock STC is coupled in common to comparators 23 and 25. Video decoding time stamp register 22 receives and stores the separated video decoding time stamp DTSV and supplies this video decoding time stamp to comparator 23. When the system time clock STC is incremented to a count equal to the video decoding time stamp DTSV (STC=DTSV), comparator 23 generates a video decoding start signal. Similarly, audio decoding time stamp register 24 receives and stores the separated audio decoding time stamp DTSA. This stored audio decoding time stamp is supplied to comparator 25; and when the system time clock STC is incremented to a count that is equal to the audio decoding time stamp DTSA (STC=DTSA), comparator 25 generates an audio decoding start signal.
Video buffer 6 (FIG. 6B) preferably is in the form of a first-in first-out (FIFO) memory and supplies separated video data to video decoder 7. When comparator 23 produces the video decoding start signal, the video decoder begins to decode the video data temporarily stored in video buffer 8.
Similarly, audio buffer 8 may be a FIFO memory; and supplies the audio data separated by data separator 21 to audio decoder 9. When comparator 25 produces the audio decoding start signal, the audio decoder is enabled to begin decoding the audio data temporarily stored in audio buffer 8.
FIG. 6A also illustrates a control circuit 28, which may be a central processing unit, coupled to data separator 21 to supply various control command signals thereto. Control circuit 28 responds to operator-generated input signals produced by an input section 29 for controlling the overall operation of the data reproducing apparatus. For example, an operator may activate input section 29 to produce those signals that are typical in optical disk drive devices, such as play, stop, pause, skip, etc. FIG. 6A illustrates that input section 29 also is coupled to disk drive 1 so as to supply such operator-generated input signals thereto.
FIG. 8 is a schematic timing diagram of the timing relationship between the operation of data separator 21 and the video and audio decoders 7 and 9. Let it be assumed that a PLAY signal is produced by input section 29, whereupon control circuit 28 supplies a demultiplexing instruction to data separator 21. Let it be further assumed that data separator 21 begins its demultiplexing operation at time t.sub.1 (FIG. 8) and, for convenience, this time t.sub.1 may be equal to the system clock reference SCR. Accordingly, clock register 26 is preset with this clock value t.sub.1. Line A in FIG. 8 illustrates the video data being written into video buffer 6; and it is appreciated that the slope of line A corresponds to the transfer rate at which such video data is written therein. Time t.sub.2 represents the video decoding time stamp DTSV. Hence, when clock register 26 is incremented from clock value t.sub.1 to clock value t.sub.2, comparator 23 supplies the video decoding start signal to video decoder 7, whereupon a unit of video data which had been temporarily stored in video buffer 6 is decoded. As shown in FIG. 8, this unit is equal to one video frame, or one video picture.
In FIG. 8, line B, which is a discontinuous line, represents the video data which is read from video buffer 6 and decoded by video decoder 7. FIG. 8 also illustrates the capacity of video buffer 6. It will be recognized that buffer overflow may occur if the rate at which video data is read from video buffer 6 is too slow, that is, it is less than the rate at which video data is written therein. Conversely, buffer underflow may occur if the contents of the video buffer are read out before a new frame is written therein. The shaded area beneath line A schematically illustrates the amount of video data remaining in video buffer 6.
Video decoder 7 produces a vertical synchronizing signal when a complete frame of video data has been decoded. If the video decoder is coupled to a suitable video display, a delay is imparted (VIDEO DECODE DELAY) to the decoded video signal prior to its display. This delayed relationship also is illustrated in FIG. 8.
It will be appreciated that the timing relationship depicted in FIG. 8 and described above in connection with the video decoding channel is equally applicable to the audio decoding channel. However, the timing relationship, referred to herein as the synchronization relationship, between video decoder 7 and audio decoder 9 is dependent upon the presetting of clock register 26 by the system clock reference SCR and the incrementing of the clock register to system clock values that are equal to the video and audio decoding time stamps, respectively. But, since the video and audio decoding time stamps are separated by data separator 21, any variations in the synchronization relationship between the video and audio data that might occur in or be attributed to the video and audio buffers will go undetected. One example of a synchronization error that may occur in the apparatus shown in FIGS. 6A and 6B and that will not be detected and, thus, not corrected, now will be described.
Let it be assumed that the data which is reproduced from the disk drive exhibits a high error rate such that ECC circuit 3 is unable to correct such errors. Referring to FIG. 9A, let it be assumed that successive picture intervals reproduced from the optical disk are pictures P12, B11, P14, B13, I1, B0, P3 and B2, wherein the numeral represents the picture interval and the letter represents the usual MPEG characterizations, namely I refers to an intraframe encoded video picture, P refers to a forward predictive encoded video picture and B refers to a bidirectionally predictive encoded video picture. Assuming no uncorrectable errors, video decoder 7 decodes the video data included in the successive picture intervals supplied thereto and rearranges the decoded data in the proper picture sequence shown in FIG. 9B. Consistent with the MPEG standard, a group of pictures (GOP) is comprised of fourteen pictures, or picture intervals, and the number of the first picture included in a GOP is reset to 0.
Let it be assumed that an uncorrectable error occurs in reproduced picture interval P14, as shown in FIG. 9C. For example, video packet header information may be lost. Since video decoder 7 cannot decode video picture P14, the video decoder simply skips this picture and, as shown in FIG. 9D, decodes picture intervals P12, B11, B13, I1, B0, P3 and B2 as if picture P14 never was present. Consequently, the output of the video decoder appears as shown in FIG. 9E. However, when FIG. 9E is compared to FIG. 9B, it is seen that the decoded picture sequence is advanced by one picture interval when an error which prevents the decoding of picture P14 is present. But, since a similar error was not present in the audio data, audio decoder 9 does not similarly advance the units of audio data decoded thereby. Hence, the decoded video and audio data now exhibit loss of synchronization therebetween.
Because data separator 21 operates to separate the video and audio decoding time stamps from the multiplexed data supplied by ring buffer 4, it is structurally difficult to reestablish synchronization between the video and audio data supplied to video and audio buffers 6 and 8 from data separator 21 when, as described above, a unit of video data is lost but a unit of audio data is not. Accordingly, to reestablish synchronization between the video and audio data in the presence of the aforementioned error requires relatively complicated processing downstream of the video and audio buffers.
Another example of a condition which requires resynchronization is described with reference to a "pause" operation, the timing relationship of which is illustrated in FIGS. 10A-10C. In the absence of errors, and during a normal play operation, video and audio data are decoded and displayed in the manner shown in FIG. 10A. As before, a unit of video data corresponds to a video picture, or picture interval, and a unit of audio data is comprised of a predetermined number of samples of the audio information (as one example, 512 samples comprise one unit of audio data). It is clearly seen that the length of one unit of video data is greater than the length of one unit of audio data. Moreover, there is no integral multiple relationship between the length of a unit of video data and the length of a unit of audio data (L.sub.v .perspectiveto.nL.sub.A, where L.sub.v is the length of a unit of video data, L.sub.A is the length of a unit of audio data and is an integral number). As is apparent from FIG. 10A, the synchronization relationship between the video and audio data thus varies continually. However, and with reference to an arbitrary picture interval, FIG. 10A illustrates a time difference t.sub.d between the beginning of picture interval P14 and the beginning of unit A10 of audio data.
Let it be assumed that the user of the reproducing apparatus shown in FIGS. 6A and 6B initiates a pause operation during the time that picture interval B13 is displayed, as illustrated in FIG. 10B. As a result of this pause operation, picture interval B13 is repeatedly displayed. Audio decoder 9 decodes units A7, A8 and A9 of audio data; but since unit A10 accompanies picture interval P14 and since picture P14 is not decoded and displayed, a muting condition exists after unit A9 of audio data is decoded. This muting condition also is illustrated in FIG. 10B and coincides with the repeated display of picture B13.
When the pause operation ends, as shown in FIG. 10C, the next picture interval P14 is displayed; and at a time t.sub.d following the beginning of picture interval P14, the next unit A10 of audio data is decoded. For proper synchronization between the video and audio data, decoding of unit A10 must begin at the delayed time t.sub.d following the beginning of picture P14.
However, in the reproducing apparatus shown in FIGS. 6A and 6B, picture interval P14 and unit A10 of audio data are separated early on from the multiplexed data reproduced from the optical disk and, likewise, the video and audio decoding time stamps are stored early on in registers 22 and 24. With the separated video picture P14 stored in video buffer 6, the separated unit A10 of audio data stored in audio buffer 8, the video decoding time stamp DTSV stored in register 22 and the audio decoding time stamp DTSA stored in register 24, the time difference t.sub.d between the beginning of video picture P14 and unit A10 of audio data is quite difficult to detect. Hence, and since this delay t.sub.d is variable, as discussed above, it is difficult to restore proper synchronization to the video and audio data at the conclusion of a pause operation.
Another drawback associated with the apparatus shown in FIGS. 6A and 6B occurs when disk drive 1 operates to re-read data from, for example, a particular sector because a high error rate has prevented the originally read data from being properly interpreted. For instance, if the disk drive is subjected to shock, vibration or mechanical interference whereby data which is read from a portion of the disk appears as an uncorrectable error, control circuit 28 may, in response to this uncorrectable error, command the disk drive to re-read the same portion of the record medium. Typically, such re-reading results in corrected data which then may be accurately decoded. For example, if the disk drive is a CD-ROM, up to a maximum of 300 milliseconds may be required to access and re-read the same portion of the disk. But, during the re-read operation, the supply of new video and audio data to video and audio buffers 6 and 8 is interrupted. Nevertheless, the data that had been stored previously therein is decoded. As a consequence, the contents of ring buffer 4, video buffer 6 and audio buffer 8 may be exhausted before new data is supplied thereto. This underflow condition may result in noticeable interruptions in the displayed video picture and audio sound; and constitutes a drawback that desirably should be avoided.